This document came about because of the System Heap/INIT paranoia I've been seeing lately. It represents the total sum of the knowledge I have accumulated from writing system level software (INITs/cdevs/System Extensions) for 3 years, and is accurate based on my experience and the experiences of many other Macintosh programmers.
It will help you make rational decisions about resolving INIT conflicts, dispel some common myths, teach you a thing or two, and tell you what is really going on with all this System Heap stuff.
Neophytes, forgive me if I confuse you with the programmer-speak necessary to explain this topic. Programmers, forgive me if I generalize issues for the sake of clarity.
INITs, cdevs, Control Panels, System Extensions -- they are all buzz-words for basically the same critters: low-level programs developers write which load automatically when the system starts up. Programs that you toss into your System Folder. From now on, I will refer to them all as INITs, though there are some subtle differences between them (not important in this discussion though).
INITs are notorious for causing system crashes, conflicts, and general system weirdness. When technical support people hear that their INIT is causing problems, you'll inevitably hear something to the effect of "Have you tried expanding your System Heap?" What do they mean when they say this, and is it the ultimate solution?
You are most likely about to learn more about the way your computer works than you want to, but if you persevere you won't regret it. Knowledge is power. Hang on tight. But first, a little background...
DIVING IN
For starters, your Macintosh has a fixed amount of memory installed in it, which acts like a desk space for things you are currently working on. In real life, when you want to work on something you might take it out of your filing cabinet and put it on your desk where you can work on it effectively. This is similar to what a computer does also: all of your programs and documents are stored on your hard disk, and when you want to work on them they are read in from disk and placed in memory.
Life being what it is, you can't have everything. You only have a limited amount of memory that must be rationed out to the programs that you run. In fact each program you run grabs a fixed chunk of memory when it loads, and it doesn't let go of the memory until you quit the program. This is the application's world to do with as it pleases. This is also known in technoweenie terms as a HEAP.
Each application is said to have its own memory HEAP. You can decide how big this HEAP is (how much memory the application will use, or how much of your desk space you're giving up to use the program). You can do this for any program by choosing "Get Info" on it in the Finder and entering how much memory you want it to use. That assumes you are running under System 7 or MultiFinder. Under older systems (sometimes called uni-Finder) the programs that you run use all of your Macintosh's memory, so you can only run one program at a time.
Well actually I just lied. Sorry about that, I know I hardly know you but I lied to you already. There is another HEAP that is always around (even if you aren't running any programs) called the SYSTEM HEAP. This is important. The SYSTEM HEAP is a chunk of memory that the System Software (the nitty gritty, low level stuff) uses for its own purposes. Things with nasty names like device drivers, ROM patch code, etc. dwell there. It is a murky, convoluted place that only Apple knows every nook and cranny of, but a very important one nevertheless.
So if you think of your computer's memory (or RAM) again, a portion of it is always occupied by the SYSTEM HEAP, an area of memory designated for the low level system software from Apple and other nasties. The rest of your computer's memory is rationed off in chunks to the applications that you run on a first come, first serve basis. Normally, we developers stay in our own little application HEAPs, insulated from the the complexities of system-level software. There are times however that the only possible way to write the programs we dream of is to delve into the world of System Software and the SYSTEM HEAP. Yes friends, we are talking about INITs (you knew I'd get around to it some day, right?).
NOW ONTO THE INITS!
If a program wants to achieve some kind of a global effects (like QuicKeys allowing you to define Macros that work in any program), they have find out how to graft themselves into your System, and keep a portion of memory for themselves that will stay around even when a program quits (remember folks, the memory a program had allocated to itself is freed up again when it is quit).
We are talking sophisticated stuff here. Writing System Software is extremely tricky stuff, even for Apple who knows every little subtle nuance in the Macintosh, it is a tough thing to do right. That is why System 7 took so long, and why Apple should be praised for System 7: it works well, which is nothing short of a miracle. But I digress.
You get the idea, writing System Software is not for the faint at heart. In any case, perhaps you guessed it: INITs take a portion of SYSTEM HEAP memory for themselves when the computer starts up. It is also then that they "hook" themselves into the System Software. When your computer first starts up, the SYSTEM HEAP is a certain fixed size. However while your machine is loading, the SYSTEM HEAP will grow automagically to accommodate an INIT that needs more memory for itself.
For technoweenies, the mechanism that loads INITs at start up time and expands the SYSTEM HEAP as needed is called INIT31. There is a well known mechanism that any INIT writer worth his salt takes advantage of: 'sysz', or System Zone Expansion. This is a little parcel that all INIT writers should include in their products (it takes all of two minutes to put it in), it tells the INIT31 mechanism how much memory the INIT needs to load. INIT31 grows the SYSTEM HEAP so that there is at least this much memory free, and then loads and runs the INIT. Therefore any well-written INIT will tell INIT31 how much memory it needs, get it, and be happy. And all is well in INIT-land.
If it were a perfect world, our story would end here. But as I'm sure you suspect, there are a few twists here. INITs have the power to affect the operation of you entire computer, and unfortunately they can also crash the whole thing too. There are a few snags that INITs can run into regarding the SYSTEM HEAP, and I will go into them one by one so that you can understand what is going wrong and why.
One thing we should get straight now: if your machine is crashing because of an INIT, there is a programmer out there somewhere who is at fault. Count on it. There are well established rules for writing INITs, the real problem is not all programmers really understand what is going on. Information on how to write INITs properly is also hard to come by. Another problem is programmers are only human, we're doing our best but we make mistakes.
THE KILLER INIT
When an INIT causes a problem, you'll hear ten people shout in unison: "Have you increased the size of your SYSTEM HEAP?" There are utilities out there that let you manually make the SYSTEM HEAP bigger, in an effort to fix crashes due to INITs. However this really isn't much of a solution. In some cases, increasing the size of your SYSTEM HEAP by 20K or so can be beneficial, so as to give the SYSTEM HEAP a little breathing room. If you still experience problems, increasing the SYSTEM HEAP more probably won't help you. It is kind of like this:
You are in a room with a sadistic murder wielding a nasty looking knife, who definitely doesn't mean well for you. Now the bigger the room you are in, the better chance you have to escape from him. Heck, if he is out of shape and the room is big enough, he may never catch you. But one day when you least expect it...
The same goes for the SYSTEM HEAP size method of "curing" crashes. Here are the common causes of problems INITs can run into, and what you can do about them:
PROBLEM #1
Many INITs are simply badly written. No amount of fudging is going to help you avoid crashes from a badly written INIT, or two INITs that don't get along together. True, increasing the size of your SYSTEM HEAP may delay the inevitable. For instance, some INITs don't even bother to check if they actually have enough memory to do what they want, and increasing your SYSTEM HEAP a little (20K or so) should help this. But if an INIT is badly written, there may be nothing you can do about it. Increasing your SYSTEM HEAP memory to try and fix a blatant programming error is just a waste of memory.
PROBLEM #2
Many INITs don't bother to have 'sysz's, thus the INIT31 mechanism has no way of knowing how much memory to give to the INIT. This is one case that increasing the size of your SYSTEM HEAP DOES INDEED cure! However the 'sysz' method has been documented by Apple or years, and I'd be suspicious of an INIT that doesn't take advantage of 'sysz' anyway.
PROBLEM #3
To understand this problem, I'll need to tell you a bit more about HEAPs. HEAPs are called HEAPs for good reason: imagine a big laundry basket into which you throw all your clothes. This is sort of the way that HEAPs work. Fortunately for programmers, there is a maid to take care of this mess called the Memory Manager. I call her Martha (just one of those things that keeps you sane dealing with something that is NEVER wrong).
Well when you want to put something in a HEAP, you ask Martha for the space for it, and she gives it to you if she can. Like all good maids, Martha knows that if the HEAP is messy, there is less room in it. So if she can't give you what you ask for immediately, she takes everything in the HEAP and cleans it up a bit to try and free up some more room. This is a nice thing! However we all know the experience of coming into your room after someone else cleaned it: you can never find anything! Many INITs have this problem as well, and it is absolutely THEIR fault, not Martha's. She can tell you where anything is, but some INITs don't bother to ask, they simply *assume* they know where their little chunk of memory is. *** crash ***
There is one other twist to this situation as well. If Martha cleans everything up, and there still isn't enough room to satisfy a request for memory, she starts throwing things out. But it again isn't her fault, these things are specifically marked "Throw me out if you need the space". If an INIT doesn't take this into account, *** crash ***
Since the SYSTEM HEAP is shared by System Software and all of your INITs, it is particularly active. However the whole scheme can and SHOULD work. Quite well in fact.
In this situation, expanding your SYSTEM HEAP can delay crashes, because if Martha never has to move things or throw things out, badly written software will work most of the time. Realize though that you are sacrificing your precious memory to correct a defect in the software you paid good money for.
PROBLEM #4
This is a tricky one that many programmers don't understand fully. Earlier I described the 'sysz' method for telling INIT31 how much memory an INIT needs to loads. And it works as advertised, but there is a rub.
The SYSTEM HEAP is shared memory, it is one big basket into which many hens put their eggs. Lets say we have an INIT called "Longcut" that has a 'sysz' specifying that it needs 50K of memory. INIT31 will free up 50K of memory, then load the hypothetical "Longcut" INIT, and all should be well.
However let's say "Longcut" doesn't use all 50K of the memory it asks for at start up time. Let's say it uses 10K of memory at start up time to install itself, reserving the rest for later on when it actually works its magic. This is a very common situation actually. Well logic would tell you that there should be 40K left over, memory that "Longcut" can count on being there, right? WRONG!
As I said before, the SYSTEM HEAP is shared memory. If an INIT asks for more memory than it actually uses at start up time, whatever it doesn't use isn't reserved for it automatically; instead it is thrown "back in the pot", and can be used up by anyone. So later on when "Longcut" thinks it has room to spare, it actually might not and then... *** crash ***
Of course there are ways to program around this situation properly, but the above isn't a well known fact. In this case, increasing SYSTEM HEAP memory can be beneficial as well, but only in a limited sense. Only the programmers know how much memory their INIT really needs, they should be the ones to fix the problem.
LITTLE KNOWN FACTS
Many people do not realize this, but with all versions of MultiFinder, the SYSTEM HEAP can actually grow even AFTER start up time, easing INIT memory conflicts. If an INIT asks for more memory than is available, the SYSTEM HEAP will automagically get bigger. However there are two bad things about this: 1) the SYSTEM HEAP will grow, but never shrink again and 2) Several well respected Macintosh programmers have stated that they've seen cases where MultiFinder CANNOT expand the SYSTEM HEAP automatically, and die in the process. This is apparently extremely rare however.
This makes claims of "you've got to increase your System Heap" even less powerful, because the System will increase it for you if need be.
System 7 changes the game entirely, the SYSTEM HEAP can grow AND shrink at will to accommodate requests for memory, eliminating many INIT memory problems entirely. However System 7 threw INIT writers several other curves, so you'll most likely need to upgrade your favorite INITs for use under System 7.0. You shouldn't have to increase your SYSTEM HEAP manually for System 7.0, but you never know... an extra 20K or so never hurt anyone.
SUMMING IT UP
PROPERLY written INITs shouldn't exhibit any of the above stated problems. But if your Macintosh needs a little tweak to get it working smoothly again, there is nothing wrong to giving a little more memory to the SYSTEM HEAP. In NO CASE should you every have to allocate more than 50K to 100K of extra memory to your SYSTEM HEAP. After a point, no amount of fudging is going to save you. Try to figure out the culprit INIT that is causing problems, and trash it. Or better yet, make sure you have the latest version of every INIT you use. Realize that INIT crashes/conflicts DO NOT stem from lack of SYSTEM HEAP memory, but from improperly written software. Giving the SYSTEM HEAP extra memory may circumvent some problems, but it is not the source of them.
When you choose "About the Finder" from the Apple Menu, there should be a little bit of white space (free memory) in the "System" memory bar. 5-10% or so should be fine.
RECOMMENDATIONS
INITs are very sensitive pieces of code, expect them to be upgraded more frequently then other software products, and keep up with the current versions. Your Macintosh will be more stable if you use the current versions of the INITs you use, and if you use INITs only from well known and established vendors who support their products. Taking these precautions will result in a far more stable Macintosh than any amount of SYSTEM HEAP tweaking ever could.
20K - 50K of extra SYSTEM HEAP space (free SYSTEM HEAP memory), OK. That is a nice margin of error. More than that? Forget it, tell them to fix it!
Free programs that can adjust SYSTEM HEAP size manually:
Heap Fixer
BootMan
Heap Tool
Contact the author at:
Andrew Welch
Mark 3 Software
29 Grey Rocks Road
Wilton, CT 06897
...and if you include a disk and a self-addressed, stamped mailer, I will send you the latest versions of my shareware programs!
